{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Untitled (2).ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/csaybar/EEwPython/blob/master/dnn_demo.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "osShnOZVo8Bw",
        "colab_type": "text"
      },
      "source": [
        "<!--COURSE_INFORMATION-->\n",
        "<img align=\"left\" style=\"padding-right:10px;\" src=\"https://sitejerk.com/images/google-earth-logo-png-5.png\" width=5% >\n",
        "<img align=\"right\" style=\"padding-left:10px;\" src=\"https://colab.research.google.com/img/colab_favicon_256px.png\" width=6% >\n",
        "\n",
        "\n",
        ">> *This notebook is part of the free course [EEwPython](https://colab.research.google.com/github/csaybar/EEwPython/blob/master/index.ipynb); the content is available [on GitHub](https://github.com/csaybar/EEwPython)* and released under the [Apache 2.0 License](https://www.gnu.org/licenses/gpl-3.0.en.html). "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fKwh6Dqxo_Ll",
        "colab_type": "text"
      },
      "source": [
        "<!--NAVIGATION-->\n",
        " < [Climate Change - CMIP5](cmip5.ipynb) | [Contents](index.ipynb) | [Crop Area Prediction Using a Convolutional Neural Network](CNN_demo.ipynb)>\n",
        "\n",
        "<a href=\"https://colab.research.google.com/github/csaybar/EEwPython/blob/master/dnn_demo.ipynb\"><img align=\"left\" src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open in Colab\" title=\"Open and Execute in Google Colaboratory\"></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Su_tIldcds-V",
        "colab_type": "text"
      },
      "source": [
        "<center>\n",
        "<h1>Google Earth Engine with Python </h1>\n",
        "<h2> Integrating Earth Engine with Tensorflow I - DNN </h2>\n",
        "</center>\n",
        "\n",
        "This notebook has been inspired by the [Chris Brown & Nick Clinton EarthEngine + Tensorflow presentation](https://www.youtube.com/watch?v=w-1xfF0IaeU). It shows the step by step how to integrate Google Earth Engine and TensorFlow 2.0 in the same pipeline (EE->Tensorflow->EE).\n",
        "\n",
        "```\n",
        "FILE: dnn_demo.ipynb\n",
        "AUTHOR: Cesar Aybar\n",
        "EMAIL: csaybar@gmail.com\n",
        "ORGANIZATION: n/a \n",
        "MODIFIED BY: n/a\n",
        "CREATION DATE: 23 May. 2019\n",
        "LAST MOD DATE: N/A\n",
        "DEPENDENCIES: earthengine-api, os, pandas, urllib, plotly, numpy, folium\n",
        "PURPOSE: Introduction to tensorflow & GEE integration\n",
        "```\n",
        "\n",
        "<center>\n",
        "<img src=\"https://raw.githubusercontent.com/csaybar/EEwPython/master/images/colab_ee_integration.png\">\n",
        "</center>  \n",
        "\n",
        "## Topics\n",
        "\n",
        "1. Create a training/testing dataset (in a TFRecord format) using Earth Engine.\n",
        "2.  Create functions for parse data (TFRecord -> tf.data.Dataset).\n",
        "3.   Training and Test a simple Vanilla Deep Neural Network using tensorflow 2.0.\n",
        "3.   Making predictions on image data exported from Earth Engine in TFRecord format.\n",
        "4.   Upload your results to Earth Engine (asset).\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "## 1. Introduction\n",
        "\n",
        "Deep learning has dramatically improved the state-of-the-art in various science domains. For remote sensing, its potential has not been thoroughly explored. This could be related to the problematic incorporation of spectral & spatial features into a regular deep learning classification scheme or the huge pre-processing that satellite images could need it. Hence, this post aims to teach you how to create a painless deep learning workflow integrating [Google Earth engine](https://earthengine.google.com/) for acquiring spectral & spatial data and [tensorflow](https://www.tensorflow.org/?hl=en) for train and test the model and make predictions.\n",
        "\n",
        "\n",
        "## 2. What is Google Earth Engine (GEE)?\n",
        "\n",
        "In a nutshell, is a platform that combines a multi-petabyte [catalog of satellite imagery](https://developers.google.com/earth-engine/datasets/) with planetary-scale analysis capabilities. There are several ways to interact with GEE:\n",
        "\n",
        "  - [Explorer](https://explorer.earthengine.google.com/)\n",
        "  - [Code Editor](https://code.earthengine.google.com/)  \n",
        "  - [Javascript wrapper library](https://github.com/google/earthengine-api/tree/master/javascript)\n",
        "  - [**Python wrapper library**](https://github.com/google/earthengine-api/tree/master/python)\n",
        "\n",
        "In my opinion, the **Python wrapper library** (used in this post) is the best choice to interact with GEE for the following reasons:\n",
        "\n",
        "- Easy to share code.\n",
        "- Easy transition to a web application.\n",
        "- Possibility to integrate with  ML/DL frameworks.\n",
        "- Many plotting options (folium, plotly, matplotlib, seaborn ,etc.).\n",
        "\n",
        "One more thing!, it's possible to run the Earth Engine Python API in a cloud environment for free. See the introduction of the [EEcourse](https://colab.research.google.com/github/csaybar/EEwPython/blob/dev/1_Introduction.ipynb) for more details.\n",
        "\n",
        "## 3.  Deep Neural Network (DNN)\n",
        "\n",
        "<center>\n",
        "<image src=\"https://www.electronicdesign.com/sites/electronicdesign.com/files/MachineLearning_WTD_Fig3.png\">\n",
        "</center>\n",
        "  \n",
        "A DNN is simply a neural network with more than two layers. The main steps for building a DNN are:\n",
        "\n",
        "1. Initialize the model's parameters.\n",
        "2. Loop:\n",
        "    - Calculate current loss (forward propagation) :    \n",
        "    - Calculate current gradient (backward propagation)\n",
        "    - Update parameters (gradient descent)\n",
        "\n",
        "The second step could be  a little intimidating, but don't worry about it!. **tf.keras**, the TensorFlow's high-level API, only need that you define the forward propagation correctly and all the steps further down will make automatically. This post does not intend to introduce the algorithm, check out this [repo](https://github.com/csaybar/DLcoursera) for a from zero (numpy) implementation.\n",
        "\n",
        "## 4. Crop Area estimation in Camana Valley (DEMO)\n",
        "\n",
        "Agriculture is part of the backbone Peruvian economy, contributing about 7.6% of the Gross Domestic Product (GDP), being more critical in rural areas where the contribution of GDP increase until 50%. In terms of people, this activity act for the primary source of income for 2.3 million families, representing 34% of Peruvian households. Despite agriculture importance in Peruvian family lives, today no exist a cropping system either at a national or regional scale that monitoring the extension, state, or crop type. Considering this problematic, in this section **you** will create a straightforward methodology to **predict the crop area** in Camana (Arequipa) Valley using a Deep Neural Network.\n",
        "\n",
        "<center>\n",
        "<img src='https://st.depositphotos.com/1171712/3974/i/950/depositphotos_39741899-stock-photo-camana-valley.jpg'>\n",
        "</center>\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "03au8DXeuIlH",
        "colab_type": "code",
        "cellView": "form",
        "colab": {}
      },
      "source": [
        "#@title Mapdisplay: Display ee.Features and ee.Images using folium.\n",
        "def Mapdisplay(center, dicc, Tiles=\"OpensTreetMap\",zoom_start=10):\n",
        "    '''\n",
        "    :param center: Center of the map (Latitude and Longitude).\n",
        "    :param dicc: Earth Engine Geometries or Tiles dictionary\n",
        "    :param Tiles: Mapbox Bright,Mapbox Control Room,Stamen Terrain,Stamen Toner,stamenwatercolor,cartodbpositron.\n",
        "    :zoom_start: Initial zoom level for the map.\n",
        "    :return: A folium.Map object.\n",
        "    '''\n",
        "    mapViz = folium.Map(location=center,tiles=Tiles, zoom_start=zoom_start)\n",
        "    for k,v in dicc.items():\n",
        "      if ee.image.Image in [type(x) for x in v.values()]:\n",
        "        folium.TileLayer(\n",
        "            tiles = EE_TILES.format(**v),\n",
        "            attr  = 'Google Earth Engine',\n",
        "            overlay =True,\n",
        "            name  = k\n",
        "          ).add_to(mapViz)\n",
        "      else:\n",
        "        folium.GeoJson(\n",
        "        data = v,\n",
        "        name = k\n",
        "          ).add_to(mapViz)\n",
        "    mapViz.add_child(folium.LayerControl())\n",
        "    return mapViz"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4YIUU4Xzfm-N",
        "colab_type": "text"
      },
      "source": [
        "### 4.1. Installing\n",
        "\n",
        "Before coding do not forget install and load the following packages and remenber that you can communicate with the bash console prepending an ! to the code. \n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nrZboinybQto",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# This tensorflow version is necessary just for tensorboard support.\n",
        "# developer version, on future it maybe breaks. \n",
        "!pip install tf-nightly-2.0-preview==2.0.0.dev20190606 \n",
        "\n",
        "!pip install earthengine-api==0.1.175 #earthengine API\n",
        "# Load the TensorBoard notebook extension\n",
        "%load_ext tensorboard"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uWnZhQhWgPjE",
        "colab_type": "text"
      },
      "source": [
        "### 4.2. Authentification\n",
        "\n",
        "This tutorial needs interacting with some Google services. For accomplish this task, it's necessary to authenticate (as yourself). The code below shows you how to do it."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fFixOP706rwY",
        "colab_type": "text"
      },
      "source": [
        "#### Google Cloud\n",
        "\n",
        "Google Cloud Storage bucket will serve as a bridge between GEE and Colab.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6EcG3BElgRa7",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from google.colab import auth\n",
        "auth.authenticate_user()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DAuPWosHgswN",
        "colab_type": "text"
      },
      "source": [
        "#### Google Earth Engine"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bxad71Q6gTXF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!earthengine authenticate"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q6_5QeN_gZh0",
        "colab_type": "text"
      },
      "source": [
        "### 4.3. Initialize and testing the software setup\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bEP-G9o7gYq0",
        "colab_type": "code",
        "outputId": "dbb2dab6-9ee2-4842-e313-f7e793be2809",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        }
      },
      "source": [
        "# Earth Engine Python API\n",
        "import ee \n",
        "ee.Initialize()\n",
        "\n",
        "import tensorflow as tf\n",
        "print('Tensorflow version: ' + tf.__version__)\n",
        "\n",
        "import folium\n",
        "print('Folium version: ' + folium.__version__)\n",
        "\n",
        "# Define the URL format used for Earth Engine generated map tiles.\n",
        "EE_TILES = 'https://earthengine.googleapis.com/map/{mapid}/{{z}}/{{x}}/{{y}}?token={token}'"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Tensorflow version: 2.0.0-dev20190606\n",
            "Folium version: 0.8.3\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FAfl6ABygYVr",
        "colab_type": "text"
      },
      "source": [
        "### 4.4. Prepare the Dataset\n",
        "\n",
        "Firstly, we define our prediction area (Camana Valley) and passing to GEE. For moving a vector to GEE, you will use the `ee.Geometry.*` module. The [GeoJSON](https://geojson.org/) spec describes in detail the type of geometries supported by GEE, including `Point` (a list of coordinates in some projection), `LineString` (a list of points), `LinearRing` (a closed LineString), and `Polygon` (a list of LinearRings where the first is a shell and subsequent rings are holes). GEE also supports **MultiPoint**, **MultiLineString**, and **MultiPolygon**. The [GeoJSON](https://geojson.org/) GeometryCollection is also supported, although it has the name **MultiGeometry** within GEE."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5dXR0HQbhVle",
        "colab_type": "code",
        "outputId": "a66bca30-9f4a-45be-e031-2f4e113b4827",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 750
        }
      },
      "source": [
        "# 2.4.1 Prediction Area\n",
        "xmin,ymin,xmax,ymax = [-72.778645, -16.621663, -72.66865, -16.57553]\n",
        "\n",
        "\n",
        "# Passing a rectangle (prediction area) to Earth Engine\n",
        "Camana_valley = ee.Geometry.Rectangle([xmin,ymin,xmax,ymax])\n",
        "center = Camana_valley.centroid().getInfo()['coordinates']\n",
        "center.reverse()\n",
        "Mapdisplay(center,{'Camana Valley':Camana_valley.getInfo()},zoom_start=12)"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div style=\"width:100%;\"><div style=\"position:relative;width:100%;height:0;padding-bottom:60%;\"><iframe src=\"data:text/html;charset=utf-8;base64,PCFET0NUWVBFIGh0bWw+CjxoZWFkPiAgICAKICAgIDxtZXRhIGh0dHAtZXF1aXY9ImNvbnRlbnQtdHlwZSIgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PVVURi04IiAvPgogICAgPHNjcmlwdD5MX1BSRUZFUl9DQU5WQVM9ZmFsc2U7IExfTk9fVE9VQ0g9ZmFsc2U7IExfRElTQUJMRV8zRD1mYWxzZTs8L3NjcmlwdD4KICAgIDxzY3JpcHQgc3JjPSJodHRwczovL2Nkbi5qc2RlbGl2ci5uZXQvbnBtL2xlYWZsZXRAMS40LjAvZGlzdC9sZWFmbGV0LmpzIj48L3NjcmlwdD4KICAgIDxzY3JpcHQgc3JjPSJodHRwczovL2NvZGUuanF1ZXJ5LmNvbS9qcXVlcnktMS4xMi40Lm1pbi5qcyI+PC9zY3JpcHQ+CiAgICA8c2NyaXB0IHNyYz0iaHR0cHM6Ly9tYXhjZG4uYm9vdHN0cmFwY2RuLmNvbS9ib290c3RyYXAvMy4yLjAvanMvYm9vdHN0cmFwLm1pbi5qcyI+PC9zY3JpcHQ+CiAgICA8c2NyaXB0IHNyYz0iaHR0cHM6Ly9jZG5qcy5jbG91ZGZsYXJlLmNvbS9hamF4L2xpYnMvTGVhZmxldC5hd2Vzb21lLW1hcmtlcnMvMi4wLjIvbGVhZmxldC5hd2Vzb21lLW1hcmtlcnMuanMiPjwvc2NyaXB0PgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL2Nkbi5qc2RlbGl2ci5uZXQvbnBtL2xlYWZsZXRAMS40LjAvZGlzdC9sZWFmbGV0LmNzcyIvPgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL21heGNkbi5ib290c3RyYXBjZG4uY29tL2Jvb3RzdHJhcC8zLjIuMC9jc3MvYm9vdHN0cmFwLm1pbi5jc3MiLz4KICAgIDxsaW5rIHJlbD0ic3R5bGVzaGVldCIgaHJlZj0iaHR0cHM6Ly9tYXhjZG4uYm9vdHN0cmFwY2RuLmNvbS9ib290c3RyYXAvMy4yLjAvY3NzL2Jvb3RzdHJhcC10aGVtZS5taW4uY3NzIi8+CiAgICA8bGluayByZWw9InN0eWxlc2hlZXQiIGhyZWY9Imh0dHBzOi8vbWF4Y2RuLmJvb3RzdHJhcGNkbi5jb20vZm9udC1hd2Vzb21lLzQuNi4zL2Nzcy9mb250LWF3ZXNvbWUubWluLmNzcyIvPgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL2NkbmpzLmNsb3VkZmxhcmUuY29tL2FqYXgvbGlicy9MZWFmbGV0LmF3ZXNvbWUtbWFya2Vycy8yLjAuMi9sZWFmbGV0LmF3ZXNvbWUtbWFya2Vycy5jc3MiLz4KICAgIDxsaW5rIHJlbD0ic3R5bGVzaGVldCIgaHJlZj0iaHR0cHM6Ly9yYXdjZG4uZ2l0aGFjay5jb20vcHl0aG9uLXZpc3VhbGl6YXRpb24vZm9saXVtL21hc3Rlci9mb2xpdW0vdGVtcGxhdGVzL2xlYWZsZXQuYXdlc29tZS5yb3RhdGUuY3NzIi8+CiAgICA8c3R5bGU+aHRtbCwgYm9keSB7d2lkdGg6IDEwMCU7aGVpZ2h0OiAxMDAlO21hcmdpbjogMDtwYWRkaW5nOiAwO308L3N0eWxlPgogICAgPHN0eWxlPiNtYXAge3Bvc2l0aW9uOmFic29sdXRlO3RvcDowO2JvdHRvbTowO3JpZ2h0OjA7bGVmdDowO308L3N0eWxlPgogICAgCiAgICA8bWV0YSBuYW1lPSJ2aWV3cG9ydCIgY29udGVudD0id2lkdGg9ZGV2aWNlLXdpZHRoLAogICAgICAgIGluaXRpYWwtc2NhbGU9MS4wLCBtYXhpbXVtLXNjYWxlPTEuMCwgdXNlci1zY2FsYWJsZT1ubyIgLz4KICAgIDxzdHlsZT4jbWFwX2IyMWUwYjUxMDlmNTQ0MTZhZjgxZjEwMGQ0NjRlZWI1IHsKICAgICAgICBwb3NpdGlvbjogcmVsYXRpdmU7CiAgICAgICAgd2lkdGg6IDEwMC4wJTsKICAgICAgICBoZWlnaHQ6IDEwMC4wJTsKICAgICAgICBsZWZ0OiAwLjAlOwogICAgICAgIHRvcDogMC4wJTsKICAgICAgICB9CiAgICA8L3N0eWxlPgo8L2hlYWQ+Cjxib2R5PiAgICAKICAgIAogICAgPGRpdiBjbGFzcz0iZm9saXVtLW1hcCIgaWQ9Im1hcF9iMjFlMGI1MTA5ZjU0NDE2YWY4MWYxMDBkNDY0ZWViNSIgPjwvZGl2Pgo8L2JvZHk+CjxzY3JpcHQ+ICAgIAogICAgCiAgICAKICAgICAgICB2YXIgYm91bmRzID0gbnVsbDsKICAgIAoKICAgIHZhciBtYXBfYjIxZTBiNTEwOWY1NDQxNmFmODFmMTAwZDQ2NGVlYjUgPSBMLm1hcCgKICAgICAgICAnbWFwX2IyMWUwYjUxMDlmNTQ0MTZhZjgxZjEwMGQ0NjRlZWI1JywgewogICAgICAgIGNlbnRlcjogWy0xNi41OTg2MDI4MDM0MTMxMSwgLTcyLjcyMzY0NzQ5OTk5OTI2XSwKICAgICAgICB6b29tOiAxMiwKICAgICAgICBtYXhCb3VuZHM6IGJvdW5kcywKICAgICAgICBsYXllcnM6IFtdLAogICAgICAgIHdvcmxkQ29weUp1bXA6IGZhbHNlLAogICAgICAgIGNyczogTC5DUlMuRVBTRzM4NTcsCiAgICAgICAgem9vbUNvbnRyb2w6IHRydWUsCiAgICAgICAgfSk7CgoKICAgIAogICAgdmFyIHRpbGVfbGF5ZXJfZWZlYWY1YjY0NDQ3NDk2OWIyYjg2MjhiNDlhOTA5NjkgPSBMLnRpbGVMYXllcigKICAgICAgICAnaHR0cHM6Ly97c30udGlsZS5vcGVuc3RyZWV0bWFwLm9yZy97en0ve3h9L3t5fS5wbmcnLAogICAgICAgIHsKICAgICAgICAiYXR0cmlidXRpb24iOiBudWxsLAogICAgICAgICJkZXRlY3RSZXRpbmEiOiBmYWxzZSwKICAgICAgICAibWF4TmF0aXZlWm9vbSI6IDE4LAogICAgICAgICJtYXhab29tIjogMTgsCiAgICAgICAgIm1pblpvb20iOiAwLAogICAgICAgICJub1dyYXAiOiBmYWxzZSwKICAgICAgICAib3BhY2l0eSI6IDEsCiAgICAgICAgInN1YmRvbWFpbnMiOiAiYWJjIiwKICAgICAgICAidG1zIjogZmFsc2UKfSkuYWRkVG8obWFwX2IyMWUwYjUxMDlmNTQ0MTZhZjgxZjEwMGQ0NjRlZWI1KTsKICAgIAogICAgICAgIHZhciBnZW9fanNvbl9iN2FjNTNkMDEwYTI0NzZlODBjNjkxYjNiMzc3MDEwNyA9IEwuZ2VvSnNvbigKICAgICAgICAgICAgeyJmZWF0dXJlcyI6IFt7Imdlb21ldHJ5IjogeyJjb29yZGluYXRlcyI6IFtbWy03Mi43Nzg2NDUsIC0xNi42MjE2NjNdLCBbLTcyLjY2ODY1LCAtMTYuNjIxNjYzXSwgWy03Mi42Njg2NSwgLTE2LjU3NTUzXSwgWy03Mi43Nzg2NDUsIC0xNi41NzU1M10sIFstNzIuNzc4NjQ1LCAtMTYuNjIxNjYzXV1dLCAidHlwZSI6ICJQb2x5Z29uIn0sICJwcm9wZXJ0aWVzIjogeyJoaWdobGlnaHQiOiB7fSwgInN0eWxlIjoge319LCAidHlwZSI6ICJGZWF0dXJlIn1dLCAidHlwZSI6ICJGZWF0dXJlQ29sbGVjdGlvbiJ9LAogICAgICAgICAgICB7CiAgICAgICAgICAgIH0KICAgICAgICApLmFkZFRvKG1hcF9iMjFlMGI1MTA5ZjU0NDE2YWY4MWYxMDBkNDY0ZWViNSApOwogICAgICAgIGdlb19qc29uX2I3YWM1M2QwMTBhMjQ3NmU4MGM2OTFiM2IzNzcwMTA3LnNldFN0eWxlKGZ1bmN0aW9uKGZlYXR1cmUpIHtyZXR1cm4gZmVhdHVyZS5wcm9wZXJ0aWVzLnN0eWxlO30pOwogICAgICAgIAogICAgCiAgICAgICAgICAgIHZhciBsYXllcl9jb250cm9sX2JiZTliNzdmMjczMjQ5M2JhMTMxOGJjMzdlZTFlOTA3ID0gewogICAgICAgICAgICAgICAgYmFzZV9sYXllcnMgOiB7ICJvcGVuc3RyZWV0bWFwIiA6IHRpbGVfbGF5ZXJfZWZlYWY1YjY0NDQ3NDk2OWIyYjg2MjhiNDlhOTA5NjksIH0sCiAgICAgICAgICAgICAgICBvdmVybGF5cyA6IHsgIkNhbWFuYSBWYWxsZXkiIDogZ2VvX2pzb25fYjdhYzUzZDAxMGEyNDc2ZTgwYzY5MWIzYjM3NzAxMDcsIH0KICAgICAgICAgICAgICAgIH07CiAgICAgICAgICAgIEwuY29udHJvbC5sYXllcnMoCiAgICAgICAgICAgICAgICBsYXllcl9jb250cm9sX2JiZTliNzdmMjczMjQ5M2JhMTMxOGJjMzdlZTFlOTA3LmJhc2VfbGF5ZXJzLAogICAgICAgICAgICAgICAgbGF5ZXJfY29udHJvbF9iYmU5Yjc3ZjI3MzI0OTNiYTEzMThiYzM3ZWUxZTkwNy5vdmVybGF5cywKICAgICAgICAgICAgICAgIHtwb3NpdGlvbjogJ3RvcHJpZ2h0JywKICAgICAgICAgICAgICAgICBjb2xsYXBzZWQ6IHRydWUsCiAgICAgICAgICAgICAgICAgYXV0b1pJbmRleDogdHJ1ZQogICAgICAgICAgICAgICAgfSkuYWRkVG8obWFwX2IyMWUwYjUxMDlmNTQ0MTZhZjgxZjEwMGQ0NjRlZWI1KTsKICAgICAgICAgICAgCiAgICAgICAgCjwvc2NyaXB0Pg==\" style=\"position:absolute;width:100%;height:100%;left:0;top:0;border:none !important;\" allowfullscreen webkitallowfullscreen mozallowfullscreen></iframe></div></div>"
            ],
            "text/plain": [
              "<folium.folium.Map at 0x7f297edfdef0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S4AEJa6sDWel",
        "colab_type": "text"
      },
      "source": [
        "Next, you will read and create a visualization of the train/test dataset. I've already generated some points with the label agriculture/non-agriculture.\n",
        "\n",
        "- Train dataset (550 points):\n",
        "  - 275 labeled as \"agriculture\"\n",
        "  - 275 labeled as \"non agriculture\" \n",
        "  \n",
        "- Test dataset (100 points):\n",
        "  - 50  labeled as \"agriculture\"\n",
        "  - 50  labeled as \"non agriculture\""
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cTcFXVBRQ8RN",
        "colab_type": "code",
        "outputId": "4dc42fd1-7a84-46e6-f26a-8fa47edf0e17",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 750
        }
      },
      "source": [
        "# 2.4.2 Importing the train/test dataset\n",
        "train_agriculture = ee.FeatureCollection('users/csaybar/DLdemos/train_set') \n",
        "test_agriculture = ee.FeatureCollection('users/csaybar/DLdemos/test_set')\n",
        "\n",
        "# Display the train/test dataset\n",
        "db_crop = train_agriculture.merge(test_agriculture)\n",
        "center = db_crop.geometry().centroid().getInfo()['coordinates']\n",
        "center.reverse()\n",
        "\n",
        "dicc = {'train': train_agriculture.draw(**{'color': 'FF0000', 'strokeWidth': 5}).getMapId(),\n",
        "        'test' : test_agriculture.draw(**{'color': '0000FF', 'strokeWidth': 5}).getMapId(),\n",
        "        'CamanaValley':Camana_valley.getInfo()\n",
        "       }\n",
        "\n",
        "Mapdisplay(center,dicc,zoom_start=8)"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div style=\"width:100%;\"><div style=\"position:relative;width:100%;height:0;padding-bottom:60%;\"><iframe src=\"data:text/html;charset=utf-8;base64,PCFET0NUWVBFIGh0bWw+CjxoZWFkPiAgICAKICAgIDxtZXRhIGh0dHAtZXF1aXY9ImNvbnRlbnQtdHlwZSIgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PVVURi04IiAvPgogICAgPHNjcmlwdD5MX1BSRUZFUl9DQU5WQVM9ZmFsc2U7IExfTk9fVE9VQ0g9ZmFsc2U7IExfRElTQUJMRV8zRD1mYWxzZTs8L3NjcmlwdD4KICAgIDxzY3JpcHQgc3JjPSJodHRwczovL2Nkbi5qc2RlbGl2ci5uZXQvbnBtL2xlYWZsZXRAMS40LjAvZGlzdC9sZWFmbGV0LmpzIj48L3NjcmlwdD4KICAgIDxzY3JpcHQgc3JjPSJodHRwczovL2NvZGUuanF1ZXJ5LmNvbS9qcXVlcnktMS4xMi40Lm1pbi5qcyI+PC9zY3JpcHQ+CiAgICA8c2NyaXB0IHNyYz0iaHR0cHM6Ly9tYXhjZG4uYm9vdHN0cmFwY2RuLmNvbS9ib290c3RyYXAvMy4yLjAvanMvYm9vdHN0cmFwLm1pbi5qcyI+PC9zY3JpcHQ+CiAgICA8c2NyaXB0IHNyYz0iaHR0cHM6Ly9jZG5qcy5jbG91ZGZsYXJlLmNvbS9hamF4L2xpYnMvTGVhZmxldC5hd2Vzb21lLW1hcmtlcnMvMi4wLjIvbGVhZmxldC5hd2Vzb21lLW1hcmtlcnMuanMiPjwvc2NyaXB0PgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL2Nkbi5qc2RlbGl2ci5uZXQvbnBtL2xlYWZsZXRAMS40LjAvZGlzdC9sZWFmbGV0LmNzcyIvPgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL21heGNkbi5ib290c3RyYXBjZG4uY29tL2Jvb3RzdHJhcC8zLjIuMC9jc3MvYm9vdHN0cmFwLm1pbi5jc3MiLz4KICAgIDxsaW5rIHJlbD0ic3R5bGVzaGVldCIgaHJlZj0iaHR0cHM6Ly9tYXhjZG4uYm9vdHN0cmFwY2RuLmNvbS9ib290c3RyYXAvMy4yLjAvY3NzL2Jvb3RzdHJhcC10aGVtZS5taW4uY3NzIi8+CiAgICA8bGluayByZWw9InN0eWxlc2hlZXQiIGhyZWY9Imh0dHBzOi8vbWF4Y2RuLmJvb3RzdHJhcGNkbi5jb20vZm9udC1hd2Vzb21lLzQuNi4zL2Nzcy9mb250LWF3ZXNvbWUubWluLmNzcyIvPgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL2NkbmpzLmNsb3VkZmxhcmUuY29tL2FqYXgvbGlicy9MZWFmbGV0LmF3ZXNvbWUtbWFya2Vycy8yLjAuMi9sZWFmbGV0LmF3ZXNvbWUtbWFya2Vycy5jc3MiLz4KICAgIDxsaW5rIHJlbD0ic3R5bGVzaGVldCIgaHJlZj0iaHR0cHM6Ly9yYXdjZG4uZ2l0aGFjay5jb20vcHl0aG9uLXZpc3VhbGl6YXRpb24vZm9saXVtL21hc3Rlci9mb2xpdW0vdGVtcGxhdGVzL2xlYWZsZXQuYXdlc29tZS5yb3RhdGUuY3NzIi8+CiAgICA8c3R5bGU+aHRtbCwgYm9keSB7d2lkdGg6IDEwMCU7aGVpZ2h0OiAxMDAlO21hcmdpbjogMDtwYWRkaW5nOiAwO308L3N0eWxlPgogICAgPHN0eWxlPiNtYXAge3Bvc2l0aW9uOmFic29sdXRlO3RvcDowO2JvdHRvbTowO3JpZ2h0OjA7bGVmdDowO308L3N0eWxlPgogICAgCiAgICA8bWV0YSBuYW1lPSJ2aWV3cG9ydCIgY29udGVudD0id2lkdGg9ZGV2aWNlLXdpZHRoLAogICAgICAgIGluaXRpYWwtc2NhbGU9MS4wLCBtYXhpbXVtLXNjYWxlPTEuMCwgdXNlci1zY2FsYWJsZT1ubyIgLz4KICAgIDxzdHlsZT4jbWFwXzQ0MGE4NzVhMzYyNDQ3NmM5MmM2ZTAzYWQxYmM2ZDRmIHsKICAgICAgICBwb3NpdGlvbjogcmVsYXRpdmU7CiAgICAgICAgd2lkdGg6IDEwMC4wJTsKICAgICAgICBoZWlnaHQ6IDEwMC4wJTsKICAgICAgICBsZWZ0OiAwLjAlOwogICAgICAgIHRvcDogMC4wJTsKICAgICAgICB9CiAgICA8L3N0eWxlPgo8L2hlYWQ+Cjxib2R5PiAgICAKICAgIAogICAgPGRpdiBjbGFzcz0iZm9saXVtLW1hcCIgaWQ9Im1hcF80NDBhODc1YTM2MjQ0NzZjOTJjNmUwM2FkMWJjNmQ0ZiIgPjwvZGl2Pgo8L2JvZHk+CjxzY3JpcHQ+ICAgIAogICAgCiAgICAKICAgICAgICB2YXIgYm91bmRzID0gbnVsbDsKICAgIAoKICAgIHZhciBtYXBfNDQwYTg3NWEzNjI0NDc2YzkyYzZlMDNhZDFiYzZkNGYgPSBMLm1hcCgKICAgICAgICAnbWFwXzQ0MGE4NzVhMzYyNDQ3NmM5MmM2ZTAzYWQxYmM2ZDRmJywgewogICAgICAgIGNlbnRlcjogWy0xNi4yMzQ4MTc3Nzc3NzMzNSwgLTcyLjQ0OTM5MzAxNzcxMjAxXSwKICAgICAgICB6b29tOiA4LAogICAgICAgIG1heEJvdW5kczogYm91bmRzLAogICAgICAgIGxheWVyczogW10sCiAgICAgICAgd29ybGRDb3B5SnVtcDogZmFsc2UsCiAgICAgICAgY3JzOiBMLkNSUy5FUFNHMzg1NywKICAgICAgICB6b29tQ29udHJvbDogdHJ1ZSwKICAgICAgICB9KTsKCgogICAgCiAgICB2YXIgdGlsZV9sYXllcl9lZDYyYTQ0OWJlNmM0MTM5OGU5MjU0OWRiZTdhYzAxYyA9IEwudGlsZUxheWVyKAogICAgICAgICdodHRwczovL3tzfS50aWxlLm9wZW5zdHJlZXRtYXAub3JnL3t6fS97eH0ve3l9LnBuZycsCiAgICAgICAgewogICAgICAgICJhdHRyaWJ1dGlvbiI6IG51bGwsCiAgICAgICAgImRldGVjdFJldGluYSI6IGZhbHNlLAogICAgICAgICJtYXhOYXRpdmVab29tIjogMTgsCiAgICAgICAgIm1heFpvb20iOiAxOCwKICAgICAgICAibWluWm9vbSI6IDAsCiAgICAgICAgIm5vV3JhcCI6IGZhbHNlLAogICAgICAgICJvcGFjaXR5IjogMSwKICAgICAgICAic3ViZG9tYWlucyI6ICJhYmMiLAogICAgICAgICJ0bXMiOiBmYWxzZQp9KS5hZGRUbyhtYXBfNDQwYTg3NWEzNjI0NDc2YzkyYzZlMDNhZDFiYzZkNGYpOwogICAgdmFyIHRpbGVfbGF5ZXJfNDczYjc1NWRmNGQzNDFiYTkzNzY2M2FlMmE1M2Q4MWUgPSBMLnRpbGVMYXllcigKICAgICAgICAnaHR0cHM6Ly9lYXJ0aGVuZ2luZS5nb29nbGVhcGlzLmNvbS9tYXAvMGUyOTc3ZWFhYzFhZDFmMjRlYTUzMDY5MWMwMzJiNDcve3p9L3t4fS97eX0/dG9rZW49ZDY4M2Y5YmZmN2Q3NDlhZGZjMzI2NDFiYzE1MGJmODUnLAogICAgICAgIHsKICAgICAgICAiYXR0cmlidXRpb24iOiAiR29vZ2xlIEVhcnRoIEVuZ2luZSIsCiAgICAgICAgImRldGVjdFJldGluYSI6IGZhbHNlLAogICAgICAgICJtYXhOYXRpdmVab29tIjogMTgsCiAgICAgICAgIm1heFpvb20iOiAxOCwKICAgICAgICAibWluWm9vbSI6IDAsCiAgICAgICAgIm5vV3JhcCI6IGZhbHNlLAogICAgICAgICJvcGFjaXR5IjogMSwKICAgICAgICAic3ViZG9tYWlucyI6ICJhYmMiLAogICAgICAgICJ0bXMiOiBmYWxzZQp9KS5hZGRUbyhtYXBfNDQwYTg3NWEzNjI0NDc2YzkyYzZlMDNhZDFiYzZkNGYpOwogICAgdmFyIHRpbGVfbGF5ZXJfMDEyZjdjN2RmYjkyNDNiNWJmODJiNTAyODhmODZkMTAgPSBMLnRpbGVMYXllcigKICAgICAgICAnaHR0cHM6Ly9lYXJ0aGVuZ2luZS5nb29nbGVhcGlzLmNvbS9tYXAvYjU2YjAwYjEyNDgzMTUxZjUxZDlhNDUwYTM4MjY1YzIve3p9L3t4fS97eX0/dG9rZW49MWUyMjkyMzRiMDlkMDMwZWU5MmY4YmQzMWVkYzZiYzcnLAogICAgICAgIHsKICAgICAgICAiYXR0cmlidXRpb24iOiAiR29vZ2xlIEVhcnRoIEVuZ2luZSIsCiAgICAgICAgImRldGVjdFJldGluYSI6IGZhbHNlLAogICAgICAgICJtYXhOYXRpdmVab29tIjogMTgsCiAgICAgICAgIm1heFpvb20iOiAxOCwKICAgICAgICAibWluWm9vbSI6IDAsCiAgICAgICAgIm5vV3JhcCI6IGZhbHNlLAogICAgICAgICJvcGFjaXR5IjogMSwKICAgICAgICAic3ViZG9tYWlucyI6ICJhYmMiLAogICAgICAgICJ0bXMiOiBmYWxzZQp9KS5hZGRUbyhtYXBfNDQwYTg3NWEzNjI0NDc2YzkyYzZlMDNhZDFiYzZkNGYpOwogICAgCiAgICAgICAgdmFyIGdlb19qc29uX2M4MGFlODYzNGY5OTQ3NmU4MzYwNjg2NmQ4YWU0MWUxID0gTC5nZW9Kc29uKAogICAgICAgICAgICB7ImZlYXR1cmVzIjogW3siZ2VvbWV0cnkiOiB7ImNvb3JkaW5hdGVzIjogW1tbLTcyLjc3ODY0NSwgLTE2LjYyMTY2M10sIFstNzIuNjY4NjUsIC0xNi42MjE2NjNdLCBbLTcyLjY2ODY1LCAtMTYuNTc1NTNdLCBbLTcyLjc3ODY0NSwgLTE2LjU3NTUzXSwgWy03Mi43Nzg2NDUsIC0xNi42MjE2NjNdXV0sICJ0eXBlIjogIlBvbHlnb24ifSwgInByb3BlcnRpZXMiOiB7ImhpZ2hsaWdodCI6IHt9LCAic3R5bGUiOiB7fX0sICJ0eXBlIjogIkZlYXR1cmUifV0sICJ0eXBlIjogIkZlYXR1cmVDb2xsZWN0aW9uIn0sCiAgICAgICAgICAgIHsKICAgICAgICAgICAgfQogICAgICAgICkuYWRkVG8obWFwXzQ0MGE4NzVhMzYyNDQ3NmM5MmM2ZTAzYWQxYmM2ZDRmICk7CiAgICAgICAgZ2VvX2pzb25fYzgwYWU4NjM0Zjk5NDc2ZTgzNjA2ODY2ZDhhZTQxZTEuc2V0U3R5bGUoZnVuY3Rpb24oZmVhdHVyZSkge3JldHVybiBmZWF0dXJlLnByb3BlcnRpZXMuc3R5bGU7fSk7CiAgICAgICAgCiAgICAKICAgICAgICAgICAgdmFyIGxheWVyX2NvbnRyb2xfZjlkYTMxYmQ3OGM2NGJlM2I1Mjc3YTIxM2FhYTQ1ZmYgPSB7CiAgICAgICAgICAgICAgICBiYXNlX2xheWVycyA6IHsgIm9wZW5zdHJlZXRtYXAiIDogdGlsZV9sYXllcl9lZDYyYTQ0OWJlNmM0MTM5OGU5MjU0OWRiZTdhYzAxYywgfSwKICAgICAgICAgICAgICAgIG92ZXJsYXlzIDogeyAidHJhaW4iIDogdGlsZV9sYXllcl80NzNiNzU1ZGY0ZDM0MWJhOTM3NjYzYWUyYTUzZDgxZSwidGVzdCIgOiB0aWxlX2xheWVyXzAxMmY3YzdkZmI5MjQzYjViZjgyYjUwMjg4Zjg2ZDEwLCJDYW1hbmFWYWxsZXkiIDogZ2VvX2pzb25fYzgwYWU4NjM0Zjk5NDc2ZTgzNjA2ODY2ZDhhZTQxZTEsIH0KICAgICAgICAgICAgICAgIH07CiAgICAgICAgICAgIEwuY29udHJvbC5sYXllcnMoCiAgICAgICAgICAgICAgICBsYXllcl9jb250cm9sX2Y5ZGEzMWJkNzhjNjRiZTNiNTI3N2EyMTNhYWE0NWZmLmJhc2VfbGF5ZXJzLAogICAgICAgICAgICAgICAgbGF5ZXJfY29udHJvbF9mOWRhMzFiZDc4YzY0YmUzYjUyNzdhMjEzYWFhNDVmZi5vdmVybGF5cywKICAgICAgICAgICAgICAgIHtwb3NpdGlvbjogJ3RvcHJpZ2h0JywKICAgICAgICAgICAgICAgICBjb2xsYXBzZWQ6IHRydWUsCiAgICAgICAgICAgICAgICAgYXV0b1pJbmRleDogdHJ1ZQogICAgICAgICAgICAgICAgfSkuYWRkVG8obWFwXzQ0MGE4NzVhMzYyNDQ3NmM5MmM2ZTAzYWQxYmM2ZDRmKTsKICAgICAgICAgICAgCiAgICAgICAgCjwvc2NyaXB0Pg==\" style=\"position:absolute;width:100%;height:100%;left:0;top:0;border:none !important;\" allowfullscreen webkitallowfullscreen mozallowfullscreen></iframe></div></div>"
            ],
            "text/plain": [
              "<folium.folium.Map at 0x7f297ee03080>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NXoIwQ4nTaxg",
        "colab_type": "text"
      },
      "source": [
        "In this part, you will obtain the input data for mapping the **Camana crop area** using  [Landsat 8 OLI/TIRS (L8)](https://developers.google.com/earth-engine/datasets/catalog/LANDSAT_LC08_C01_T1_SR). GEE provides L8 images with radiometric and geometry correction. Additionally, **cloud mask information** is provided by means of the bit image `pixel_qa`. The following function allows putting NA to the TOA reflectance values of clouds.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "U01lsbmchIgC",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def maskS2clouds(img):\n",
        "  '''  \n",
        "  Function to mask clouds based on the pixel_qa band of Landsat 8 SR data. See:\n",
        "  https://developers.google.com/earth-engine/datasets/catalog/LANDSAT_LC08_C01_T1_SR\n",
        "  \n",
        "  Params:\n",
        "  -------\n",
        "  - img: image input Landsat 8 SR image\n",
        "  \n",
        "  Return:\n",
        "  -------\n",
        "  cloudmasked Landsat 8 image\n",
        "  '''\n",
        "  cloudShadowBitMask = (1 << 3)\n",
        "  cloudsBitMask = (1 << 5)\n",
        "  # Get the pixel QA band.\n",
        "  qa = img.select('pixel_qa')\n",
        "  # Both flags should be set to zero, indicating clear conditions.\n",
        "  mask = qa.bitwiseAnd(cloudShadowBitMask).eq(0)\\\n",
        "           .And(qa.bitwiseAnd(cloudsBitMask).eq(0))\n",
        "  return img.updateMask(mask)     "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R1LIVmdsKg4E",
        "colab_type": "text"
      },
      "source": [
        "Now you will filter and reduce the entire Landsat-8 dataset, considering the following:\n",
        "\n",
        "1. Select just bands **[R, G, B, NIR]**.\n",
        "\n",
        "2. Filter considering the cloud pixel percentage by scene (< 20%).\n",
        "\n",
        "3. Filter considering a date (we just selecting 1 years)\n",
        "\n",
        "4. Apply **mask2cloud** to each image.\n",
        "\n",
        "5. Get the median of the ImageCollection.\n",
        "\n",
        "6. Clip the image considering study area.\n",
        "\n",
        "**NOTE:** To apply a function on all the elements of specified **`ImageCollection`** or **`FeatureCollection`**, you can use the **`map()`** function. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Y12K-bnbKhO9",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# 2.4.3 Prepare the satellite image (Landsat-8)\n",
        "RGB_bands = ['B4','B3','B2'] #RGB\n",
        "NDVI_bands = ['B5','B4'] #NIR\n",
        "\n",
        "l8 = ee.ImageCollection(\"LANDSAT/LC08/C01/T1_SR\")\\\n",
        "               .filterBounds(db_crop)\\\n",
        "               .filterDate('2018-01-01', '2018-12-31')\\\n",
        "               .filter(ee.Filter.lt('CLOUD_COVER', 20))\\\n",
        "               .map(maskS2clouds)\\\n",
        "               .median()\\\n",
        "               .multiply(0.0001)\n",
        "\n",
        "l8_ndvi = l8.normalizedDifference(NDVI_bands).rename(['NDVI'])\n",
        "l8_rgb = l8.select(RGB_bands).rename(['R','G','B']) \n",
        "l8 = l8_rgb.addBands(l8_ndvi)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a4ixJRVEVMjj",
        "colab_type": "code",
        "outputId": "af8892f4-a095-4c2d-bb4f-59d9e91276f4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 750
        }
      },
      "source": [
        "from collections import OrderedDict\n",
        "# Create a visualization with folium\n",
        "visParams_l8 = {    \n",
        "  'bands': ['R', 'G', 'B'],\n",
        "  'min': 0,\n",
        "  'max': 0.5,\n",
        "  'gamma': 1.4,\n",
        "}\n",
        "\n",
        "l8Mapid = l8.getMapId(visParams_l8)\n",
        "dicc['Landsat8'] = l8Mapid\n",
        "\n",
        "# Changing the order of the dictionary\n",
        "key_order = ['Landsat8','CamanaValley','train','test']\n",
        "dicc = OrderedDict((k, dicc[k]) for k in key_order)\n",
        "\n",
        "Mapdisplay(center,dicc,zoom_start=8)"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div style=\"width:100%;\"><div style=\"position:relative;width:100%;height:0;padding-bottom:60%;\"><iframe src=\"data:text/html;charset=utf-8;base64,PCFET0NUWVBFIGh0bWw+CjxoZWFkPiAgICAKICAgIDxtZXRhIGh0dHAtZXF1aXY9ImNvbnRlbnQtdHlwZSIgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PVVURi04IiAvPgogICAgPHNjcmlwdD5MX1BSRUZFUl9DQU5WQVM9ZmFsc2U7IExfTk9fVE9VQ0g9ZmFsc2U7IExfRElTQUJMRV8zRD1mYWxzZTs8L3NjcmlwdD4KICAgIDxzY3JpcHQgc3JjPSJodHRwczovL2Nkbi5qc2RlbGl2ci5uZXQvbnBtL2xlYWZsZXRAMS40LjAvZGlzdC9sZWFmbGV0LmpzIj48L3NjcmlwdD4KICAgIDxzY3JpcHQgc3JjPSJodHRwczovL2NvZGUuanF1ZXJ5LmNvbS9qcXVlcnktMS4xMi40Lm1pbi5qcyI+PC9zY3JpcHQ+CiAgICA8c2NyaXB0IHNyYz0iaHR0cHM6Ly9tYXhjZG4uYm9vdHN0cmFwY2RuLmNvbS9ib290c3RyYXAvMy4yLjAvanMvYm9vdHN0cmFwLm1pbi5qcyI+PC9zY3JpcHQ+CiAgICA8c2NyaXB0IHNyYz0iaHR0cHM6Ly9jZG5qcy5jbG91ZGZsYXJlLmNvbS9hamF4L2xpYnMvTGVhZmxldC5hd2Vzb21lLW1hcmtlcnMvMi4wLjIvbGVhZmxldC5hd2Vzb21lLW1hcmtlcnMuanMiPjwvc2NyaXB0PgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL2Nkbi5qc2RlbGl2ci5uZXQvbnBtL2xlYWZsZXRAMS40LjAvZGlzdC9sZWFmbGV0LmNzcyIvPgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL21heGNkbi5ib290c3RyYXBjZG4uY29tL2Jvb3RzdHJhcC8zLjIuMC9jc3MvYm9vdHN0cmFwLm1pbi5jc3MiLz4KICAgIDxsaW5rIHJlbD0ic3R5bGVzaGVldCIgaHJlZj0iaHR0cHM6Ly9tYXhjZG4uYm9vdHN0cmFwY2RuLmNvbS9ib290c3RyYXAvMy4yLjAvY3NzL2Jvb3RzdHJhcC10aGVtZS5taW4uY3NzIi8+CiAgICA8bGluayByZWw9InN0eWxlc2hlZXQiIGhyZWY9Imh0dHBzOi8vbWF4Y2RuLmJvb3RzdHJhcGNkbi5jb20vZm9udC1hd2Vzb21lLzQuNi4zL2Nzcy9mb250LWF3ZXNvbWUubWluLmNzcyIvPgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL2NkbmpzLmNsb3VkZmxhcmUuY29tL2FqYXgvbGlicy9MZWFmbGV0LmF3ZXNvbWUtbWFya2Vycy8yLjAuMi9sZWFmbGV0LmF3ZXNvbWUtbWFya2Vycy5jc3MiLz4KICAgIDxsaW5rIHJlbD0ic3R5bGVzaGVldCIgaHJlZj0iaHR0cHM6Ly9yYXdjZG4uZ2l0aGFjay5jb20vcHl0aG9uLXZpc3VhbGl6YXRpb24vZm9saXVtL21hc3Rlci9mb2xpdW0vdGVtcGxhdGVzL2xlYWZsZXQuYXdlc29tZS5yb3RhdGUuY3NzIi8+CiAgICA8c3R5bGU+aHRtbCwgYm9keSB7d2lkdGg6IDEwMCU7aGVpZ2h0OiAxMDAlO21hcmdpbjogMDtwYWRkaW5nOiAwO308L3N0eWxlPgogICAgPHN0eWxlPiNtYXAge3Bvc2l0aW9uOmFic29sdXRlO3RvcDowO2JvdHRvbTowO3JpZ2h0OjA7bGVmdDowO308L3N0eWxlPgogICAgCiAgICA8bWV0YSBuYW1lPSJ2aWV3cG9ydCIgY29udGVudD0id2lkdGg9ZGV2aWNlLXdpZHRoLAogICAgICAgIGluaXRpYWwtc2NhbGU9MS4wLCBtYXhpbXVtLXNjYWxlPTEuMCwgdXNlci1zY2FsYWJsZT1ubyIgLz4KICAgIDxzdHlsZT4jbWFwXzNlMmJjN2NkMzZhMjQxNTc5YWRkM2NkNmJlYTVkMTg3IHsKICAgICAgICBwb3NpdGlvbjogcmVsYXRpdmU7CiAgICAgICAgd2lkdGg6IDEwMC4wJTsKICAgICAgICBoZWlnaHQ6IDEwMC4wJTsKICAgICAgICBsZWZ0OiAwLjAlOwogICAgICAgIHRvcDogMC4wJTsKICAgICAgICB9CiAgICA8L3N0eWxlPgo8L2hlYWQ+Cjxib2R5PiAgICAKICAgIAogICAgPGRpdiBjbGFzcz0iZm9saXVtLW1hcCIgaWQ9Im1hcF8zZTJiYzdjZDM2YTI0MTU3OWFkZDNjZDZiZWE1ZDE4NyIgPjwvZGl2Pgo8L2JvZHk+CjxzY3JpcHQ+ICAgIAogICAgCiAgICAKICAgICAgICB2YXIgYm91bmRzID0gbnVsbDsKICAgIAoKICAgIHZhciBtYXBfM2UyYmM3Y2QzNmEyNDE1NzlhZGQzY2Q2YmVhNWQxODcgPSBMLm1hcCgKICAgICAgICAnbWFwXzNlMmJjN2NkMzZhMjQxNTc5YWRkM2NkNmJlYTVkMTg3JywgewogICAgICAgIGNlbnRlcjogWy0xNi4yMzQ4MTc3Nzc3NzMzNSwgLTcyLjQ0OTM5MzAxNzcxMjAxXSwKICAgICAgICB6b29tOiA4LAogICAgICAgIG1heEJvdW5kczogYm91bmRzLAogICAgICAgIGxheWVyczogW10sCiAgICAgICAgd29ybGRDb3B5SnVtcDogZmFsc2UsCiAgICAgICAgY3JzOiBMLkNSUy5FUFNHMzg1NywKICAgICAgICB6b29tQ29udHJvbDogdHJ1ZSwKICAgICAgICB9KTsKCgogICAgCiAgICB2YXIgdGlsZV9sYXllcl9kYzk0ZTI3NzE5N2I0Yzc1OTBiNjYzM2VlODVmZWQ4OCA9IEwudGlsZUxheWVyKAogICAgICAgICdodHRwczovL3tzfS50aWxlLm9wZW5zdHJlZXRtYXAub3JnL3t6fS97eH0ve3l9LnBuZycsCiAgICAgICAgewogICAgICAgICJhdHRyaWJ1dGlvbiI6IG51bGwsCiAgICAgICAgImRldGVjdFJldGluYSI6IGZhbHNlLAogICAgICAgICJtYXhOYXRpdmVab29tIjogMTgsCiAgICAgICAgIm1heFpvb20iOiAxOCwKICAgICAgICAibWluWm9vbSI6IDAsCiAgICAgICAgIm5vV3JhcCI6IGZhbHNlLAogICAgICAgICJvcGFjaXR5IjogMSwKICAgICAgICAic3ViZG9tYWlucyI6ICJhYmMiLAogICAgICAgICJ0bXMiOiBmYWxzZQp9KS5hZGRUbyhtYXBfM2UyYmM3Y2QzNmEyNDE1NzlhZGQzY2Q2YmVhNWQxODcpOwogICAgdmFyIHRpbGVfbGF5ZXJfOGFmMzFkZmFkMzM4NDQyZTlkMTkzODkzNWMwNWY3Y2MgPSBMLnRpbGVMYXllcigKICAgICAgICAnaHR0cHM6Ly9lYXJ0aGVuZ2luZS5nb29nbGVhcGlzLmNvbS9tYXAvMzRkMzUzNDBkYTllNjBkNjZmNGE5MzcxNzMwYjhiMjQve3p9L3t4fS97eX0/dG9rZW49OThkMDJmYTE0NzBlOWRhZWU1YWZhNzBiMTg2NWMzYzInLAogICAgICAgIHsKICAgICAgICAiYXR0cmlidXRpb24iOiAiR29vZ2xlIEVhcnRoIEVuZ2luZSIsCiAgICAgICAgImRldGVjdFJldGluYSI6IGZhbHNlLAogICAgICAgICJtYXhOYXRpdmVab29tIjogMTgsCiAgICAgICAgIm1heFpvb20iOiAxOCwKICAgICAgICAibWluWm9vbSI6IDAsCiAgICAgICAgIm5vV3JhcCI6IGZhbHNlLAogICAgICAgICJvcGFjaXR5IjogMSwKICAgICAgICAic3ViZG9tYWlucyI6ICJhYmMiLAogICAgICAgICJ0bXMiOiBmYWxzZQp9KS5hZGRUbyhtYXBfM2UyYmM3Y2QzNmEyNDE1NzlhZGQzY2Q2YmVhNWQxODcpOwogICAgCiAgICAgICAgdmFyIGdlb19qc29uXzY4MTU0NzJjYTQ3ZjRiMTY4NGYxMmQwYmVmMWYyNGI0ID0gTC5nZW9Kc29uKAogICAgICAgICAgICB7ImZlYXR1cmVzIjogW3siZ2VvbWV0cnkiOiB7ImNvb3JkaW5hdGVzIjogW1tbLTcyLjc3ODY0NSwgLTE2LjYyMTY2M10sIFstNzIuNjY4NjUsIC0xNi42MjE2NjNdLCBbLTcyLjY2ODY1LCAtMTYuNTc1NTNdLCBbLTcyLjc3ODY0NSwgLTE2LjU3NTUzXSwgWy03Mi43Nzg2NDUsIC0xNi42MjE2NjNdXV0sICJ0eXBlIjogIlBvbHlnb24ifSwgInByb3BlcnRpZXMiOiB7ImhpZ2hsaWdodCI6IHt9LCAic3R5bGUiOiB7fX0sICJ0eXBlIjogIkZlYXR1cmUifV0sICJ0eXBlIjogIkZlYXR1cmVDb2xsZWN0aW9uIn0sCiAgICAgICAgICAgIHsKICAgICAgICAgICAgfQogICAgICAgICkuYWRkVG8obWFwXzNlMmJjN2NkMzZhMjQxNTc5YWRkM2NkNmJlYTVkMTg3ICk7CiAgICAgICAgZ2VvX2pzb25fNjgxNTQ3MmNhNDdmNGIxNjg0ZjEyZDBiZWYxZjI0YjQuc2V0U3R5bGUoZnVuY3Rpb24oZmVhdHVyZSkge3JldHVybiBmZWF0dXJlLnByb3BlcnRpZXMuc3R5bGU7fSk7CiAgICAgICAgCiAgICB2YXIgdGlsZV9sYXllcl9kNjEwNjcyYjg1YjI0MTQ5ODRlOTU5YzcwZjZlZjhjMiA9IEwudGlsZUxheWVyKAogICAgICAgICdodHRwczovL2VhcnRoZW5naW5lLmdvb2dsZWFwaXMuY29tL21hcC8wZTI5NzdlYWFjMWFkMWYyNGVhNTMwNjkxYzAzMmI0Ny97en0ve3h9L3t5fT90b2tlbj1kNjgzZjliZmY3ZDc0OWFkZmMzMjY0MWJjMTUwYmY4NScsCiAgICAgICAgewogICAgICAgICJhdHRyaWJ1dGlvbiI6ICJHb29nbGUgRWFydGggRW5naW5lIiwKICAgICAgICAiZGV0ZWN0UmV0aW5hIjogZmFsc2UsCiAgICAgICAgIm1heE5hdGl2ZVpvb20iOiAxOCwKICAgICAgICAibWF4Wm9vbSI6IDE4LAogICAgICAgICJtaW5ab29tIjogMCwKICAgICAgICAibm9XcmFwIjogZmFsc2UsCiAgICAgICAgIm9wYWNpdHkiOiAxLAogICAgICAgICJzdWJkb21haW5zIjogImFiYyIsCiAgICAgICAgInRtcyI6IGZhbHNlCn0pLmFkZFRvKG1hcF8zZTJiYzdjZDM2YTI0MTU3OWFkZDNjZDZiZWE1ZDE4Nyk7CiAgICB2YXIgdGlsZV9sYXllcl9iMjAzZTgzZmZkZjk0MjRiOWFjOTExNTRkZDE0NmMxNCA9IEwudGlsZUxheWVyKAogICAgICAgICdodHRwczovL2VhcnRoZW5naW5lLmdvb2dsZWFwaXMuY29tL21hcC9iNTZiMDBiMTI0ODMxNTFmNTFkOWE0NTBhMzgyNjVjMi97en0ve3h9L3t5fT90b2tlbj0xZTIyOTIzNGIwOWQwMzBlZTkyZjhiZDMxZWRjNmJjNycsCiAgICAgICAgewogICAgICAgICJhdHRyaWJ1dGlvbiI6ICJHb29nbGUgRWFydGggRW5naW5lIiwKICAgICAgICAiZGV0ZWN0UmV0aW5hIjogZmFsc2UsCiAgICAgICAgIm1heE5hdGl2ZVpvb20iOiAxOCwKICAgICAgICAibWF4Wm9vbSI6IDE4LAogICAgICAgICJtaW5ab29tIjogMCwKICAgICAgICAibm9XcmFwIjogZmFsc2UsCiAgICAgICAgIm9wYWNpdHkiOiAxLAogICAgICAgICJzdWJkb21haW5zIjogImFiYyIsCiAgICAgICAgInRtcyI6IGZhbHNlCn0pLmFkZFRvKG1hcF8zZTJiYzdjZDM2YTI0MTU3OWFkZDNjZDZiZWE1ZDE4Nyk7CiAgICAKICAgICAgICAgICAgdmFyIGxheWVyX2NvbnRyb2xfMmFiMzNlNzExY2IwNDM3NmI2ZGRmYjEyMGE4NWUyN2EgPSB7CiAgICAgICAgICAgICAgICBiYXNlX2xheWVycyA6IHsgIm9wZW5zdHJlZXRtYXAiIDogdGlsZV9sYXllcl9kYzk0ZTI3NzE5N2I0Yzc1OTBiNjYzM2VlODVmZWQ4OCwgfSwKICAgICAgICAgICAgICAgIG92ZXJsYXlzIDogeyAiTGFuZHNhdDgiIDogdGlsZV9sYXllcl84YWYzMWRmYWQzMzg0NDJlOWQxOTM4OTM1YzA1ZjdjYywiQ2FtYW5hVmFsbGV5IiA6IGdlb19qc29uXzY4MTU0NzJjYTQ3ZjRiMTY4NGYxMmQwYmVmMWYyNGI0LCJ0cmFpbiIgOiB0aWxlX2xheWVyX2Q2MTA2NzJiODViMjQxNDk4NGU5NTljNzBmNmVmOGMyLCJ0ZXN0IiA6IHRpbGVfbGF5ZXJfYjIwM2U4M2ZmZGY5NDI0YjlhYzkxMTU0ZGQxNDZjMTQsIH0KICAgICAgICAgICAgICAgIH07CiAgICAgICAgICAgIEwuY29udHJvbC5sYXllcnMoCiAgICAgICAgICAgICAgICBsYXllcl9jb250cm9sXzJhYjMzZTcxMWNiMDQzNzZiNmRkZmIxMjBhODVlMjdhLmJhc2VfbGF5ZXJzLAogICAgICAgICAgICAgICAgbGF5ZXJfY29udHJvbF8yYWIzM2U3MTFjYjA0Mzc2YjZkZGZiMTIwYTg1ZTI3YS5vdmVybGF5cywKICAgICAgICAgICAgICAgIHtwb3NpdGlvbjogJ3RvcHJpZ2h0JywKICAgICAgICAgICAgICAgICBjb2xsYXBzZWQ6IHRydWUsCiAgICAgICAgICAgICAgICAgYXV0b1pJbmRleDogdHJ1ZQogICAgICAgICAgICAgICAgfSkuYWRkVG8obWFwXzNlMmJjN2NkMzZhMjQxNTc5YWRkM2NkNmJlYTVkMTg3KTsKICAgICAgICAgICAgCiAgICAgICAgCjwvc2NyaXB0Pg==\" style=\"position:absolute;width:100%;height:100%;left:0;top:0;border:none !important;\" allowfullscreen webkitallowfullscreen mozallowfullscreen></iframe></div></div>"
            ],
            "text/plain": [
              "<folium.folium.Map at 0x7f297ed0dd68>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JAiq7cTHJQsX",
        "colab_type": "text"
      },
      "source": [
        "For the collocation of the train/test datasets to each L8 grid cell value you will use the `ee.Image.sampleRegions` function. Note that geometries will be snapped to pixel centers (nearest center approximation).\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cWpTWAvvbXh0",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# 2.4.3 Extract pixels values considering train/test dataset\n",
        "train_db = l8.sampleRegions(collection=train_agriculture, properties=['class'], scale=30)\n",
        "test_db = l8.sampleRegions(collection=test_agriculture, properties=['class'], scale=30)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "-Zuyg3ndS0_R"
      },
      "source": [
        "Unfortunately, you cannot use Tensorflow directly in Earth Engine. To overcome this problem, you will save the train/test dataset into a Google **Cloud Storage Bucket (GCS)** (you could use Google Drive instead), since both GEE and GCS and Tensorflow can access to them. More details about how to export data in GEE you can find them in the next [link](10_Export.ipynb) or into the [Official Exporting data guide](https://developers.google.com/earth-engine/exporting)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "e6HzmpE7bXXz",
        "colab_type": "code",
        "outputId": "3b262c15-3f13-4f09-c992-372eaf8810cb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "# 2.4.4 Save results in Google Cloud Storage\n",
        "outputBucket = 'bag_csaybar' # Replace with your Cloud Storage bucket \n",
        "\n",
        "# Make sure the bucket exists.\n",
        "print('Found Cloud Storage bucket.' if tf.io.gfile.exists('gs://' + outputBucket) \n",
        "    else 'Output Cloud Storage bucket does not exist.')\n",
        "\n",
        "trainFilePrefix = 'TrainingAgriculture_'\n",
        "testFilePrefix = 'TestAgriculture_'\n",
        "\n",
        "# Create the tasks for passing of GEE to Google storage\n",
        "trainingTask = ee.batch.Export.table.toCloudStorage(\n",
        "  collection=train_db,\n",
        "  description='Training Export',\n",
        "  fileNamePrefix=trainFilePrefix,\n",
        "  bucket=outputBucket,\n",
        "  fileFormat='TFRecord')\n",
        "\n",
        "testingTask = ee.batch.Export.table.toCloudStorage(\n",
        "  collection=test_db,\n",
        "  description='Testing Export',\n",
        "  fileNamePrefix=testFilePrefix,\n",
        "  bucket=outputBucket,\n",
        "  fileFormat='TFRecord')\n",
        "\n",
        "trainingTask.start()\n",
        "testingTask.start()"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Found Cloud Storage bucket.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MJRMmCaDcItW",
        "colab_type": "code",
        "outputId": "8d8c4a57-5028-4f49-e088-66680c7139b1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 125
        }
      },
      "source": [
        "# Monitor task progress\n",
        "# Code Extracted here:\n",
        "# https://github.com/google/earthengine-api/blob/master/python/examples/ipynb/TF_demo1_keras.ipynb\n",
        "import time \n",
        "while trainingTask.active():\n",
        "  print('Polling for task (id: {}).'.format(trainingTask.id))\n",
        "  time.sleep(5)\n",
        "while testingTask.active():\n",
        "  print('Polling for task (id: {}).'.format(testingTask.id))\n",
        "  time.sleep(5)\n",
        "\n",
        "print('Done!')"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Polling for task (id: VPBH2OPHEXI6EFSCSJDT2RHT).\n",
            "Polling for task (id: VPBH2OPHEXI6EFSCSJDT2RHT).\n",
            "Polling for task (id: VPBH2OPHEXI6EFSCSJDT2RHT).\n",
            "Polling for task (id: VPBH2OPHEXI6EFSCSJDT2RHT).\n",
            "Polling for task (id: VPBH2OPHEXI6EFSCSJDT2RHT).\n",
            "Done!\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BeFmmhoKdngc",
        "colab_type": "text"
      },
      "source": [
        "### 4.5. Creating a tf.data.Dataset from a TFRecord file\n",
        "\n",
        "Read data from the TFRecord file into a tf.data.Dataset. Pre-process the dataset to get it into a suitable format for input to the DNN model. \n",
        "For getting more details about  `tf.data.Dataset`see the next [TFdoc](https://www.tensorflow.org/guide/premade_estimators#create_input_functions).\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8szRwsIxf7dc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Fullname train/test db\n",
        "fileNameSuffix = 'ee_export.tfrecord.gz'\n",
        "trainFilePath = 'gs://' + outputBucket + '/' + trainFilePrefix + fileNameSuffix\n",
        "testFilePath = 'gs://' + outputBucket + '/' + testFilePrefix + fileNameSuffix"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RxN8c3C4f7LX",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def input_fn(fileNames, numEpochs=None, shuffle=True, batchSize=16):\n",
        "  # Read `TFRecordDatasets` \n",
        "  dataset = tf.data.TFRecordDataset(fileNames, compression_type='GZIP')\n",
        "\n",
        "  # Names of the features \n",
        "  feature_columns = {\n",
        "    'R': tf.io.FixedLenFeature([], dtype=tf.float32),  \n",
        "    'G': tf.io.FixedLenFeature([], dtype=tf.float32),  \n",
        "    'B': tf.io.FixedLenFeature([], dtype=tf.float32),    \n",
        "    'NDVI': tf.io.FixedLenFeature([], dtype=tf.float32),    \n",
        "    'class': tf.io.FixedLenFeature([], dtype=tf.float32)\n",
        "  }\n",
        "  \n",
        "  # Make a parsing function\n",
        "  def parse(example_proto):\n",
        "    parsed_features = tf.io.parse_single_example(example_proto, feature_columns)    \n",
        "    # Separate the class labels from the training features\n",
        "    labels = parsed_features.pop('class')\n",
        "    return parsed_features, tf.cast(labels, tf.int32)\n",
        "  \n",
        "  # Map the function over the dataset\n",
        "  dataset = dataset.map(parse, num_parallel_calls=5)\n",
        "  if shuffle:\n",
        "    dataset = dataset.shuffle(buffer_size = batchSize * 10)\n",
        "  dataset = dataset.batch(batchSize)\n",
        "  dataset = dataset.repeat(numEpochs)\n",
        "  \n",
        "  return dataset"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DU3l1Lj9ls7d",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "train_dba = input_fn(trainFilePath,100,True,32)\n",
        "test_dba = input_fn(testFilePath, numEpochs=1, batchSize=1, shuffle=False)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Koi4qoKSrH79",
        "colab_type": "text"
      },
      "source": [
        "### 4.6. Create a DNN model with keras\n",
        "\n",
        "Here you will create a deep neural network model with:\n",
        "- 2 layers (10x10 nodes).\n",
        "- 1 dropout layer.\n",
        "- 1 output layer.\n",
        "\n",
        "Additionally, Early Stopping, Tensorboard, and best model callback were added. A callback is a set of functions to be applied at given stages of the training procedure. You can found more details [here](https://keras.io/callbacks/).\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eiKMSYnhO1g3",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 667
        },
        "outputId": "305a4bc8-aeeb-4c22-ed17-b985ff75dfbf"
      },
      "source": [
        "from tensorflow import keras\n",
        "from tensorflow.keras import layers\n",
        "from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint\n",
        "from tensorflow import feature_column\n",
        "import os\n",
        "import datetime\n",
        "\n",
        "\n",
        "bands = ['R','G','B','NDVI']\n",
        "\n",
        "# Create a dense `Tensor` based on given `feature_columns`.\n",
        "feature_columns = [feature_column.numeric_column(x) for x in bands]\n",
        "feature_layer = tf.keras.layers.DenseFeatures(feature_columns)\n",
        "\n",
        "# Initialize the DNN model\n",
        "he_init = tf.keras.initializers.he_uniform(seed=None)\n",
        "\n",
        "# Define the layers in the model.\n",
        "model = tf.keras.Sequential([\n",
        "  feature_layer,  \n",
        "  layers.Dense(10, activation='relu',kernel_initializer=he_init),\n",
        "  tf.keras.layers.Dropout(0.2),\n",
        "  layers.Dense(10, activation='relu',kernel_initializer=he_init),\n",
        "  layers.Dense(1, activation='sigmoid',kernel_initializer=he_init)\n",
        "])\n",
        "\n",
        "# Callbacks time\n",
        "logdir = os.path.join(\"logs\", datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\"))\n",
        "tensorboard_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)\n",
        "es = EarlyStopping(monitor='val_loss', patience=10)\n",
        "mcp = ModelCheckpoint(filepath='best_model.h5', monitor='val_loss', save_best_only=True)\n",
        "\n",
        "\n",
        "# Compile the model with the specified loss function.\n",
        "model.compile(optimizer=keras.optimizers.SGD(momentum=0.01, nesterov=True),\n",
        "              loss='binary_crossentropy',              \n",
        "              metrics=['accuracy'])\n",
        "\n",
        "# Fit the model to the training data.\n",
        "model.fit(x=train_dba,\n",
        "          epochs=15,\n",
        "          steps_per_epoch=100,\n",
        "          callbacks=[tensorboard_callback,es,mcp],\n",
        "          validation_data=test_dba)"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/15\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING: Logging before flag parsing goes to stderr.\n",
            "W0608 00:46:57.023546 139817096210304 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_grad.py:1251: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Use tf.where in 2.0, which has the same broadcast rule as np.where\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "100/100 [==============================] - 3s 25ms/step - loss: 0.6591 - accuracy: 0.6542 - val_loss: 0.6196 - val_accuracy: 0.9500\n",
            "Epoch 2/15\n",
            "100/100 [==============================] - 0s 5ms/step - loss: 0.6167 - accuracy: 0.7760 - val_loss: 0.5709 - val_accuracy: 0.9400\n",
            "Epoch 3/15\n",
            "100/100 [==============================] - 1s 5ms/step - loss: 0.5851 - accuracy: 0.8135 - val_loss: 0.5285 - val_accuracy: 0.9300\n",
            "Epoch 4/15\n",
            "100/100 [==============================] - 0s 5ms/step - loss: 0.5535 - accuracy: 0.8224 - val_loss: 0.4882 - val_accuracy: 0.9400\n",
            "Epoch 5/15\n",
            "100/100 [==============================] - 0s 5ms/step - loss: 0.5248 - accuracy: 0.8280 - val_loss: 0.4500 - val_accuracy: 0.9500\n",
            "Epoch 6/15\n",
            "100/100 [==============================] - 0s 5ms/step - loss: 0.5039 - accuracy: 0.8268 - val_loss: 0.4166 - val_accuracy: 0.9500\n",
            "Epoch 7/15\n",
            "100/100 [==============================] - 0s 5ms/step - loss: 0.4814 - accuracy: 0.8071 - val_loss: 0.3875 - val_accuracy: 0.9500\n",
            "Epoch 8/15\n",
            "100/100 [==============================] - 1s 5ms/step - loss: 0.4576 - accuracy: 0.8378 - val_loss: 0.3603 - val_accuracy: 0.9500\n",
            "Epoch 9/15\n",
            "100/100 [==============================] - 1s 6ms/step - loss: 0.4457 - accuracy: 0.8225 - val_loss: 0.3380 - val_accuracy: 0.9400\n",
            "Epoch 10/15\n",
            "100/100 [==============================] - 0s 5ms/step - loss: 0.4288 - accuracy: 0.8350 - val_loss: 0.3188 - val_accuracy: 0.9300\n",
            "Epoch 11/15\n",
            "100/100 [==============================] - 1s 6ms/step - loss: 0.4231 - accuracy: 0.8322 - val_loss: 0.3015 - val_accuracy: 0.9400\n",
            "Epoch 12/15\n",
            "100/100 [==============================] - 0s 5ms/step - loss: 0.4086 - accuracy: 0.8336 - val_loss: 0.2870 - val_accuracy: 0.9300\n",
            "Epoch 13/15\n",
            "100/100 [==============================] - 0s 5ms/step - loss: 0.4020 - accuracy: 0.8107 - val_loss: 0.2776 - val_accuracy: 0.9300\n",
            "Epoch 14/15\n",
            "100/100 [==============================] - 0s 5ms/step - loss: 0.3966 - accuracy: 0.8332 - val_loss: 0.2654 - val_accuracy: 0.9300\n",
            "Epoch 15/15\n",
            "100/100 [==============================] - 0s 5ms/step - loss: 0.3855 - accuracy: 0.8301 - val_loss: 0.2560 - val_accuracy: 0.9300\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow.python.keras.callbacks.History at 0x7f2979be2ba8>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "es8Y2pVpYXM8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%tensorboard --logdir logs\n",
        "#!kill 607"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0jJtpiYtVfwz",
        "colab_type": "code",
        "outputId": "f5761065-802d-4646-9beb-7c8ba0842f6d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "model.load_weights(\"best_model.h5\")\n",
        "model.evaluate(x = test_dba)"
      ],
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "    100/Unknown - 1s 5ms/step - loss: 0.2560 - accuracy: 0.9300"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[0.25602747334167364, 0.93]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l08s8WgjhIEj",
        "colab_type": "text"
      },
      "source": [
        "### 4.7. Prediction\n",
        "\n",
        "You will prepare the L8 imagery, likewise, you made it for the train/test dataset. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iKe0xaMchKO_",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "l8 = ee.ImageCollection(\"LANDSAT/LC08/C01/T1_SR\")\\\n",
        "               .filterBounds(Camana_valley)\\\n",
        "               .filterDate('2018-01-01', '2018-12-31')\\\n",
        "               .filter(ee.Filter.lt('CLOUD_COVER', 20))\\\n",
        "               .map(maskS2clouds)\\\n",
        "               .median()\\\n",
        "               .multiply(0.0001)\\\n",
        "               .clip(Camana_valley)\n",
        "\n",
        "l8_ndvi = l8.normalizedDifference(NDVI_bands).rename(['NDVI'])\n",
        "l8_rgb = l8.select(RGB_bands).rename(['R','G','B']) \n",
        "l8 = l8_rgb.addBands(l8_ndvi)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sv4kCrptYos1",
        "colab_type": "text"
      },
      "source": [
        "For export the results to the Google Cloud Storage, it's preferred defines the following `formatOptions` parameters to save memory:\n",
        "\n",
        "- **patchDimensions**:\tPatch dimensions tiled over the export area, covering every pixel in the bounding box exactly once (except when the patch dimensions do not evenly divide the bounding box in which case the lower and right sides are trimmed).\n",
        "\n",
        "- **compressed**: If true, compresses the .tfrecord files with gzip and appends the \".gz\" suffix\t\n",
        "\n",
        "See all the paramerters [here](https://developers.google.com/earth-engine/exporting#configuration-parameters)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "unxNYJeAkT0V",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "imageFilePrefix = 'CamanaValleyCrop'\n",
        "\n",
        "# Specify patch and file dimensions.\n",
        "imageExportFormatOptions = {\n",
        "  'patchDimensions': [64, 64],  \n",
        "  'maxFileSize': 100000000,    \n",
        "  'compressed': True\n",
        "}\n",
        "\n",
        "# Setup the task.\n",
        "imageTask = ee.batch.Export.image.toCloudStorage(\n",
        "  image=l8,\n",
        "  description='Image Export',\n",
        "  fileNamePrefix=imageFilePrefix,\n",
        "  bucket=outputBucket,\n",
        "  scale=30,\n",
        "  fileFormat='TFRecord',\n",
        "  region=Camana_valley.getInfo()['coordinates'],\n",
        "  formatOptions=imageExportFormatOptions,\n",
        ")\n",
        "\n",
        "imageTask.start()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "JmXlRNgCl7YS",
        "colab_type": "code",
        "outputId": "242ab342-3fcb-408f-da3b-02895ed3cd5a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 226
        }
      },
      "source": [
        "import time \n",
        "while imageTask.active():\n",
        "  print('Polling for task (id: {}).'.format(imageTask.id))\n",
        "  time.sleep(5)"
      ],
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "error",
          "ename": "NameError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-27-6b47f4669ade>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0;32mwhile\u001b[0m \u001b[0mimageTask\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mactive\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m   \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Polling for task (id: {}).'\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mimageTask\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mid\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m   \u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msleep\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m5\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mNameError\u001b[0m: name 'imageTask' is not defined"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p3KgV33i0wvR",
        "colab_type": "text"
      },
      "source": [
        "Now it's time to classify the image that was exported from GEE to GCS using Tensorflow. If the exported image is large (it not your case), it will be split into multiple TFRecord files in its destination folder. There will also be a JSON sidecar file called **\"the mixer\"** that describes the format and georeferencing of the image. Here we will find the image files and the mixer file, getting some info out of the mixer that will be useful during model inference."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Dnsh-wb8mAyT",
        "colab_type": "code",
        "outputId": "50b7ce80-7454-4433-d301-4d8a703b0660",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "filesList = !gsutil ls 'gs://'{outputBucket}\n",
        "exportFilesList = [s for s in filesList if imageFilePrefix in s]\n",
        "\n",
        "# Get the list of image files and the JSON mixer file.\n",
        "imageFilesList = []\n",
        "jsonFile = None\n",
        "for f in exportFilesList:\n",
        "  if f.endswith('.tfrecord.gz'):\n",
        "    imageFilesList.append(f)\n",
        "  elif f.endswith('.json'):\n",
        "    jsonFile = f\n",
        "\n",
        "# Make sure the files are in the right order.\n",
        "print(jsonFile)"
      ],
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "gs://bag_csaybar/CamanaValleyCrop.json\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n53qwrAfbpq8",
        "colab_type": "text"
      },
      "source": [
        "The mixer contains metadata and georeferencing information for the exported patches, each of which is in a different file. Read the mixer to get some information needed for prediction."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XNnZ6W-9m50c",
        "colab_type": "code",
        "outputId": "6cbb3508-5f8d-4d4d-d161-0c7eb5152230",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 197
        }
      },
      "source": [
        "import json\n",
        "from pprint import pprint \n",
        "\n",
        "# Load the contents of the mixer file to a JSON object.\n",
        "jsonText = !gsutil cat {jsonFile}\n",
        "# Get a single string w/ newlines from the IPython.utils.text.SList\n",
        "mixer = json.loads(jsonText.nlstr)\n",
        "pprint(mixer)"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "{'patchDimensions': [64, 64],\n",
            " 'patchesPerRow': 6,\n",
            " 'projection': {'affine': {'doubleMatrix': [0.00026949458523585647,\n",
            "                                            0.0,\n",
            "                                            -72.77889920503969,\n",
            "                                            0.0,\n",
            "                                            -0.00026949458523585647,\n",
            "                                            -16.57526446493135]},\n",
            "                'crs': 'EPSG:4326'},\n",
            " 'totalPatches': 12}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PlSm0_2DdC5g",
        "colab_type": "text"
      },
      "source": [
        "The next function  is slightly different from the to the `input_fn` (see Section 4.5). Mainly, this is because the pixels are written into records as patches, we need to read the patches in as one big tensor (one patch for each band), then flatten them into lots of little tensors.  Once the `predict_input_fn` is defined, that can handle the shape of the image data, all you need to do is feed it directly to the trained model to make predictions.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "b0DR0wlim83X",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def predict_input_fn(fileNames,patch,bands):\n",
        "\n",
        "  # You have to know the following from your export.\n",
        "  PATCH_WIDTH, PATCH_HEIGHT = patch\n",
        "  PATCH_DIMENSIONS_FLAT = [PATCH_WIDTH * PATCH_HEIGHT, 1]\n",
        "\n",
        "  # Note that the tensors are in the shape of a patch, one patch for each band.\n",
        "  imageColumns = [\n",
        "    tf.io.FixedLenFeature(shape=PATCH_DIMENSIONS_FLAT, dtype=tf.float32) \n",
        "      for k in bands\n",
        "  ]\n",
        "\n",
        "  featuresDict = dict(zip(bands, imageColumns))\n",
        "  dataset = tf.data.TFRecordDataset(fileNames, compression_type='GZIP')\n",
        "  \n",
        "  # Make a parsing function\n",
        "  def parse_image(example_proto):\n",
        "    parsed_features = tf.io.parse_single_example(example_proto, featuresDict)\n",
        "    return parsed_features\n",
        "\n",
        "  dataset = dataset.map(parse_image, num_parallel_calls=4)\n",
        "  \n",
        "  # Break our long tensors into many littler ones\n",
        "  #https://stackoverflow.com/questions/50530806/using-flat-map-in-tensorflows-dataset-api\n",
        "  dataset = dataset.flat_map(lambda features: tf.data.Dataset.from_tensor_slices(features))\n",
        "\n",
        "  # Read in batches corresponding to patch size.\n",
        "  dataset = dataset.batch(PATCH_WIDTH * PATCH_HEIGHT)\n",
        "\n",
        "  return dataset"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WnRFSIpCfjD4",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "predict_db = predict_input_fn(fileNames=imageFilesList,patch=[64,64],bands=['R', 'G', 'B', 'NDVI'])\n",
        "predictions = model.predict(predict_db)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bmHyVSkSlrtY",
        "colab_type": "text"
      },
      "source": [
        "Now that there's a `np.array` of probabilities in \"predictions\", it's time to write them back into a file. You will write directly from TensorFlow to a file in the output Cloud Storage bucket.\n",
        "\n",
        "Iterate over the list and write the probabilities in patches. Specifically, we need to write the pixels into the file as patches in the same order they came out. The records are written as serialized tf.train.Example protos. This might take a while."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SdepemIOVnD-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Instantiate the writer.\n",
        "PATCH_WIDTH , PATCH_HEIGHT = [64,64]\n",
        "outputImageFile = 'gs://' + outputBucket + '/CamanaValleyCrop.TFRecord'\n",
        "writer = tf.io.TFRecordWriter(outputImageFile)\n",
        "\n",
        "# Every patch-worth of predictions we'll dump an example into the output\n",
        "# file with a single feature that holds our predictions. Since our predictions\n",
        "# are already in the order of the exported data, the patches we create here\n",
        "# will also be in the right order.\n",
        "patch = []\n",
        "curPatch = 1\n",
        "for  prediction in predictions:\n",
        "  patch.append(prediction)\n",
        "  \n",
        "  if (len(patch) == PATCH_WIDTH * PATCH_HEIGHT):\n",
        "    print('Done with patch ' + str(curPatch) + '...')    \n",
        "    # Create an example\n",
        "    example = tf.train.Example(\n",
        "      features=tf.train.Features(\n",
        "        feature={\n",
        "          'crop_prob': tf.train.Feature(\n",
        "              float_list=tf.train.FloatList(\n",
        "                  value=patch))\n",
        "        }\n",
        "      )\n",
        "    )\n",
        "    \n",
        "    writer.write(example.SerializeToString())\n",
        "    patch = []\n",
        "    curPatch += 1 \n",
        "\n",
        "writer.close()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rkO1ubuBpWYH",
        "colab_type": "text"
      },
      "source": [
        "### 4.8 Upload the classifications to an Earth Engine asset\n",
        "\n",
        "At this stage, there should be a predictions TFRecord file sitting in the output Cloud Storage bucket. Use the gsutil command to verify that the predictions image (and associated mixer JSON) exist and have non-zero size.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vlgDx1cGp0WM",
        "colab_type": "code",
        "outputId": "e260dfaf-c1a1-4e15-c5e1-cf1405b58dc2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        }
      },
      "source": [
        "!gsutil ls -l {outputImageFile}"
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "    197172  2019-06-08T00:49:17Z  gs://bag_csaybar/CamanaValleyCrop.TFRecord\n",
            "TOTAL: 1 objects, 197172 bytes (192.55 KiB)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V_fFjHiup86J",
        "colab_type": "text"
      },
      "source": [
        "Upload the image to Earth Engine directly from the Cloud Storage bucket with the [earthengine command](https://developers.google.com/earth-engine/command_line#upload). Provide both the image TFRecord file and the JSON file as arguments to earthengine upload.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mqB8MS6oWixT",
        "colab_type": "code",
        "outputId": "7df05ee0-fb31-4c4a-b2dc-d35f4afd7485",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "# REPLACE WITH YOUR USERNAME:\n",
        "USER_NAME = 'csaybar'\n",
        "outputAssetID = 'users/' + USER_NAME + '/CamanaCrop'\n",
        "print('Writing to ' + outputAssetID)"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Writing to users/csaybar/CamanaCrop\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2LLazXQgWn85",
        "colab_type": "code",
        "outputId": "a96ccdeb-02c8-4cd3-c04d-fc1fff75874a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "# Start the upload. It step might take a while.\n",
        "!earthengine upload image --asset_id={outputAssetID} {outputImageFile} {jsonFile}"
      ],
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Started upload task with ID: RNBAHAB727R546JYIHARCJUM\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZHrFVwo7qr7Y",
        "colab_type": "text"
      },
      "source": [
        "Display the Results using Folium!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jbfPqdqLbfig",
        "colab_type": "code",
        "outputId": "61819c3d-320a-4f1a-b970-a66f245914f6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 750
        }
      },
      "source": [
        "ProbsImage = ee.Image(outputAssetID)\n",
        "predictionsImage = ee.Image(outputAssetID).gte(0.505)\n",
        "dicc = {'CropProbability':ProbsImage.getMapId(),\n",
        "        'Crop':predictionsImage.getMapId()}\n",
        "\n",
        "center = Camana_valley.centroid().getInfo()['coordinates']\n",
        "center.reverse()\n",
        "\n",
        "Mapdisplay(center=center,dicc=dicc,zoom_start=13)"
      ],
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div style=\"width:100%;\"><div style=\"position:relative;width:100%;height:0;padding-bottom:60%;\"><iframe src=\"data:text/html;charset=utf-8;base64,PCFET0NUWVBFIGh0bWw+CjxoZWFkPiAgICAKICAgIDxtZXRhIGh0dHAtZXF1aXY9ImNvbnRlbnQtdHlwZSIgY29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PVVURi04IiAvPgogICAgPHNjcmlwdD5MX1BSRUZFUl9DQU5WQVM9ZmFsc2U7IExfTk9fVE9VQ0g9ZmFsc2U7IExfRElTQUJMRV8zRD1mYWxzZTs8L3NjcmlwdD4KICAgIDxzY3JpcHQgc3JjPSJodHRwczovL2Nkbi5qc2RlbGl2ci5uZXQvbnBtL2xlYWZsZXRAMS40LjAvZGlzdC9sZWFmbGV0LmpzIj48L3NjcmlwdD4KICAgIDxzY3JpcHQgc3JjPSJodHRwczovL2NvZGUuanF1ZXJ5LmNvbS9qcXVlcnktMS4xMi40Lm1pbi5qcyI+PC9zY3JpcHQ+CiAgICA8c2NyaXB0IHNyYz0iaHR0cHM6Ly9tYXhjZG4uYm9vdHN0cmFwY2RuLmNvbS9ib290c3RyYXAvMy4yLjAvanMvYm9vdHN0cmFwLm1pbi5qcyI+PC9zY3JpcHQ+CiAgICA8c2NyaXB0IHNyYz0iaHR0cHM6Ly9jZG5qcy5jbG91ZGZsYXJlLmNvbS9hamF4L2xpYnMvTGVhZmxldC5hd2Vzb21lLW1hcmtlcnMvMi4wLjIvbGVhZmxldC5hd2Vzb21lLW1hcmtlcnMuanMiPjwvc2NyaXB0PgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL2Nkbi5qc2RlbGl2ci5uZXQvbnBtL2xlYWZsZXRAMS40LjAvZGlzdC9sZWFmbGV0LmNzcyIvPgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL21heGNkbi5ib290c3RyYXBjZG4uY29tL2Jvb3RzdHJhcC8zLjIuMC9jc3MvYm9vdHN0cmFwLm1pbi5jc3MiLz4KICAgIDxsaW5rIHJlbD0ic3R5bGVzaGVldCIgaHJlZj0iaHR0cHM6Ly9tYXhjZG4uYm9vdHN0cmFwY2RuLmNvbS9ib290c3RyYXAvMy4yLjAvY3NzL2Jvb3RzdHJhcC10aGVtZS5taW4uY3NzIi8+CiAgICA8bGluayByZWw9InN0eWxlc2hlZXQiIGhyZWY9Imh0dHBzOi8vbWF4Y2RuLmJvb3RzdHJhcGNkbi5jb20vZm9udC1hd2Vzb21lLzQuNi4zL2Nzcy9mb250LWF3ZXNvbWUubWluLmNzcyIvPgogICAgPGxpbmsgcmVsPSJzdHlsZXNoZWV0IiBocmVmPSJodHRwczovL2NkbmpzLmNsb3VkZmxhcmUuY29tL2FqYXgvbGlicy9MZWFmbGV0LmF3ZXNvbWUtbWFya2Vycy8yLjAuMi9sZWFmbGV0LmF3ZXNvbWUtbWFya2Vycy5jc3MiLz4KICAgIDxsaW5rIHJlbD0ic3R5bGVzaGVldCIgaHJlZj0iaHR0cHM6Ly9yYXdjZG4uZ2l0aGFjay5jb20vcHl0aG9uLXZpc3VhbGl6YXRpb24vZm9saXVtL21hc3Rlci9mb2xpdW0vdGVtcGxhdGVzL2xlYWZsZXQuYXdlc29tZS5yb3RhdGUuY3NzIi8+CiAgICA8c3R5bGU+aHRtbCwgYm9keSB7d2lkdGg6IDEwMCU7aGVpZ2h0OiAxMDAlO21hcmdpbjogMDtwYWRkaW5nOiAwO308L3N0eWxlPgogICAgPHN0eWxlPiNtYXAge3Bvc2l0aW9uOmFic29sdXRlO3RvcDowO2JvdHRvbTowO3JpZ2h0OjA7bGVmdDowO308L3N0eWxlPgogICAgCiAgICA8bWV0YSBuYW1lPSJ2aWV3cG9ydCIgY29udGVudD0id2lkdGg9ZGV2aWNlLXdpZHRoLAogICAgICAgIGluaXRpYWwtc2NhbGU9MS4wLCBtYXhpbXVtLXNjYWxlPTEuMCwgdXNlci1zY2FsYWJsZT1ubyIgLz4KICAgIDxzdHlsZT4jbWFwXzRlYzVjMmJhYmZkZDQ0YTViN2MyZjE0NTI4NDZjY2UzIHsKICAgICAgICBwb3NpdGlvbjogcmVsYXRpdmU7CiAgICAgICAgd2lkdGg6IDEwMC4wJTsKICAgICAgICBoZWlnaHQ6IDEwMC4wJTsKICAgICAgICBsZWZ0OiAwLjAlOwogICAgICAgIHRvcDogMC4wJTsKICAgICAgICB9CiAgICA8L3N0eWxlPgo8L2hlYWQ+Cjxib2R5PiAgICAKICAgIAogICAgPGRpdiBjbGFzcz0iZm9saXVtLW1hcCIgaWQ9Im1hcF80ZWM1YzJiYWJmZGQ0NGE1YjdjMmYxNDUyODQ2Y2NlMyIgPjwvZGl2Pgo8L2JvZHk+CjxzY3JpcHQ+ICAgIAogICAgCiAgICAKICAgICAgICB2YXIgYm91bmRzID0gbnVsbDsKICAgIAoKICAgIHZhciBtYXBfNGVjNWMyYmFiZmRkNDRhNWI3YzJmMTQ1Mjg0NmNjZTMgPSBMLm1hcCgKICAgICAgICAnbWFwXzRlYzVjMmJhYmZkZDQ0YTViN2MyZjE0NTI4NDZjY2UzJywgewogICAgICAgIGNlbnRlcjogWy0xNi41OTg2MDI4MDM0MTMxMSwgLTcyLjcyMzY0NzQ5OTk5OTI2XSwKICAgICAgICB6b29tOiAxMywKICAgICAgICBtYXhCb3VuZHM6IGJvdW5kcywKICAgICAgICBsYXllcnM6IFtdLAogICAgICAgIHdvcmxkQ29weUp1bXA6IGZhbHNlLAogICAgICAgIGNyczogTC5DUlMuRVBTRzM4NTcsCiAgICAgICAgem9vbUNvbnRyb2w6IHRydWUsCiAgICAgICAgfSk7CgoKICAgIAogICAgdmFyIHRpbGVfbGF5ZXJfMjNlNDE5Y2U2Mzg3NDI0OWJkMGFiYTg2N2Y3MjQ3ZDEgPSBMLnRpbGVMYXllcigKICAgICAgICAnaHR0cHM6Ly97c30udGlsZS5vcGVuc3RyZWV0bWFwLm9yZy97en0ve3h9L3t5fS5wbmcnLAogICAgICAgIHsKICAgICAgICAiYXR0cmlidXRpb24iOiBudWxsLAogICAgICAgICJkZXRlY3RSZXRpbmEiOiBmYWxzZSwKICAgICAgICAibWF4TmF0aXZlWm9vbSI6IDE4LAogICAgICAgICJtYXhab29tIjogMTgsCiAgICAgICAgIm1pblpvb20iOiAwLAogICAgICAgICJub1dyYXAiOiBmYWxzZSwKICAgICAgICAib3BhY2l0eSI6IDEsCiAgICAgICAgInN1YmRvbWFpbnMiOiAiYWJjIiwKICAgICAgICAidG1zIjogZmFsc2UKfSkuYWRkVG8obWFwXzRlYzVjMmJhYmZkZDQ0YTViN2MyZjE0NTI4NDZjY2UzKTsKICAgIHZhciB0aWxlX2xheWVyX2UyMGRmYzk0ZjIyODQ2ZGI4ZWFlMDc5ODZiY2I4ZjE1ID0gTC50aWxlTGF5ZXIoCiAgICAgICAgJ2h0dHBzOi8vZWFydGhlbmdpbmUuZ29vZ2xlYXBpcy5jb20vbWFwL2QwZjNhYWI0MTAxOWU5NTQwMzI0NmQwNzg3NTI0ZTY4L3t6fS97eH0ve3l9P3Rva2VuPTRjM2FkOWRlY2MxYjMxMWM0OWI2ZGQ3MDAyOWFiYWE5JywKICAgICAgICB7CiAgICAgICAgImF0dHJpYnV0aW9uIjogIkdvb2dsZSBFYXJ0aCBFbmdpbmUiLAogICAgICAgICJkZXRlY3RSZXRpbmEiOiBmYWxzZSwKICAgICAgICAibWF4TmF0aXZlWm9vbSI6IDE4LAogICAgICAgICJtYXhab29tIjogMTgsCiAgICAgICAgIm1pblpvb20iOiAwLAogICAgICAgICJub1dyYXAiOiBmYWxzZSwKICAgICAgICAib3BhY2l0eSI6IDEsCiAgICAgICAgInN1YmRvbWFpbnMiOiAiYWJjIiwKICAgICAgICAidG1zIjogZmFsc2UKfSkuYWRkVG8obWFwXzRlYzVjMmJhYmZkZDQ0YTViN2MyZjE0NTI4NDZjY2UzKTsKICAgIHZhciB0aWxlX2xheWVyXzExN2Y5MjlmNTU0MDRjZGFhZGQ2YTkyY2Y3ZTRhNzIxID0gTC50aWxlTGF5ZXIoCiAgICAgICAgJ2h0dHBzOi8vZWFydGhlbmdpbmUuZ29vZ2xlYXBpcy5jb20vbWFwL2M3NzgxZDUzZWZlNzA0MjljMDg0NjI0NDAyYTFiNTVlL3t6fS97eH0ve3l9P3Rva2VuPWU2N2MzZjM2NTQwZDRmMGRlMzRmMzljN2Q4ZjgwMTdkJywKICAgICAgICB7CiAgICAgICAgImF0dHJpYnV0aW9uIjogIkdvb2dsZSBFYXJ0aCBFbmdpbmUiLAogICAgICAgICJkZXRlY3RSZXRpbmEiOiBmYWxzZSwKICAgICAgICAibWF4TmF0aXZlWm9vbSI6IDE4LAogICAgICAgICJtYXhab29tIjogMTgsCiAgICAgICAgIm1pblpvb20iOiAwLAogICAgICAgICJub1dyYXAiOiBmYWxzZSwKICAgICAgICAib3BhY2l0eSI6IDEsCiAgICAgICAgInN1YmRvbWFpbnMiOiAiYWJjIiwKICAgICAgICAidG1zIjogZmFsc2UKfSkuYWRkVG8obWFwXzRlYzVjMmJhYmZkZDQ0YTViN2MyZjE0NTI4NDZjY2UzKTsKICAgIAogICAgICAgICAgICB2YXIgbGF5ZXJfY29udHJvbF8xNTU3ZTdiZGY0NWI0OTQ4YjFmOTI2ZWMyNGI1YzE0ZSA9IHsKICAgICAgICAgICAgICAgIGJhc2VfbGF5ZXJzIDogeyAib3BlbnN0cmVldG1hcCIgOiB0aWxlX2xheWVyXzIzZTQxOWNlNjM4NzQyNDliZDBhYmE4NjdmNzI0N2QxLCB9LAogICAgICAgICAgICAgICAgb3ZlcmxheXMgOiB7ICJDcm9wUHJvYmFiaWxpdHkiIDogdGlsZV9sYXllcl9lMjBkZmM5NGYyMjg0NmRiOGVhZTA3OTg2YmNiOGYxNSwiQ3JvcCIgOiB0aWxlX2xheWVyXzExN2Y5MjlmNTU0MDRjZGFhZGQ2YTkyY2Y3ZTRhNzIxLCB9CiAgICAgICAgICAgICAgICB9OwogICAgICAgICAgICBMLmNvbnRyb2wubGF5ZXJzKAogICAgICAgICAgICAgICAgbGF5ZXJfY29udHJvbF8xNTU3ZTdiZGY0NWI0OTQ4YjFmOTI2ZWMyNGI1YzE0ZS5iYXNlX2xheWVycywKICAgICAgICAgICAgICAgIGxheWVyX2NvbnRyb2xfMTU1N2U3YmRmNDViNDk0OGIxZjkyNmVjMjRiNWMxNGUub3ZlcmxheXMsCiAgICAgICAgICAgICAgICB7cG9zaXRpb246ICd0b3ByaWdodCcsCiAgICAgICAgICAgICAgICAgY29sbGFwc2VkOiB0cnVlLAogICAgICAgICAgICAgICAgIGF1dG9aSW5kZXg6IHRydWUKICAgICAgICAgICAgICAgIH0pLmFkZFRvKG1hcF80ZWM1YzJiYWJmZGQ0NGE1YjdjMmYxNDUyODQ2Y2NlMyk7CiAgICAgICAgICAgIAogICAgICAgIAo8L3NjcmlwdD4=\" style=\"position:absolute;width:100%;height:100%;left:0;top:0;border:none !important;\" allowfullscreen webkitallowfullscreen mozallowfullscreen></iframe></div></div>"
            ],
            "text/plain": [
              "<folium.folium.Map at 0x7f297edff2b0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 36
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hdKdlFI0DhuF",
        "colab_type": "text"
      },
      "source": [
        "### That's all for this time!, the next post is about semantic segmentation and Earth Engine."
      ]
    }
  ]
}